The reference to any prior art in this specification is not, and should not be taken as, an acknowledgment or any form of suggestion that the prior art forms part of the common general knowledge.
Handwriting recognition systems are generally classed as writer-dependent, meaning they have been trained to recognise the specific writing style of a single user, or writer-independent, meaning they have been trained to recognise the writing of any user. Writer-dependent systems are usually able to produce more accurate recognition (for a specific user) than writer-independent systems, since they need only model the style of a single writer. However, they generally require the user to enter a large amount of training information to allow the user-specific training to take place. Conversely, writer-independent systems do not require user-specific training as they are generally developed using a corpus of training data from a large number of users. Since these systems must cater for a wide range of stylistic variation in stroke and letter formation, they are more likely to encounter confusion between character classes.
Writer adaptation is the process of transforming a writer-independent recognition system into a writer-dependent system using data provided by the user. Adaptive systems are attractive since they do not require the user to perform the lengthy process of entering training data, and can, over time, approach the recognition accuracy of a writer-dependent system. In addition to this, they are able to adapt to the changing writing style of a user over time.
One of the difficulties in developing an adaptive system is to ensure that the adaptations that occur lead to improved recognition. If not carefully implemented, adaptive procedures can decrease the overall recognition rate for a specific user, for example, by the inclusion of an incorrectly labelled prototype, or adaptation using a badly formed or ambiguous pattern. As a result, many adaptive systems require guidance from the writer to perform adaptation, using interaction with the user to ensure the data used for adaptation is well-formed and correctly labelled. Examples of this are described in U.S. Pat. No. 5,917,924, U.S. Pat. No. 5,754,686, U.S. Pat. No. 5,544,260, and U.S. Pat. No. 5,550,930.
While this can reduce the occurrence of degenerative adaptation, it requires the user to be involved with performing a number of possibly lengthy and tedious operations.
Adaptive classifiers must use some kind of learning process to allow the system to adapt to a user's specific style. Supervised learning is the process whereby the behaviour of a classifier is modified based on the correct labelling of a set of samples (ie. the correct category of each example is supplied). However, this information is generally not available to an adaptive system without user intervention, since the only labelling information available is the output of the classifier itself. Alternatively, unsupervised learning techniques (also known as self-organised learning) do not require labelled samples for the learning processes, and thus are suited to adaptive systems where the correct result is not known.
Competitive learning is an unsupervised learning process that requires elements of a system to compete with each other for activation, and is described for example in C. von der Malsburg, “Self-Organisation of Orientation Sensitive Cells in the Striate Cortex”. Kybernetik, 14:85-100, 1973, for the self-organisation of orientation-sensitive nerve cells. Similarly, it is also described in K. Fukushima, “Cognitron: a Self-Organising Multilayered Neural Network”. Biological Cybernetics, 20:121-136, 1975 for the self-organisation of a multi-layer neural network called the cognitron. There are also neurobiological justifications for competitive learning techniques as described in J. Ambros-Ingerson, R. Granger, and G. Lynch, “Simulation of Paleocortex Performs Hierarchical Clustering”. Science, 247: 1344-1348, 1990.
Further examples of documents describing these techniques will now be described. V. Vuori, J. Laaksonen, E. Oja and J. Kangas, “On-line Adaptation In Recognition of Handwritten Alphanumeric Characters”, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99. IEEE Computer Society, Los Alamitos, Calif., USA describe a user-specific adaptive system for handwritten alphanumeric characters that includes various combinations of three strategies. These include adding prototypes based on a k-NN search, inactivating prototypes which appear more harmful than useful, and prototype reshaping, based on Learning Vector Quantisation, as described in T. Kohonen, “Self Organising Maps”, Volume 30 of Springer Series in Information Sciences. Springer-Verlag, 1997.
The classifier is based on the simple Condensed Nearest-Neighbour rule, with a semi-automatic prototype-clustering algorithm used during training to condense the prototypes. Dynamic Time-Warping using various point-to-point, point-to-line, and area metrics is used to calculate the similarity measure between input and prototypes. The approach to prototype deactivation uses supervised learning (“user-reported misclassifications are used to revise the system”, as described in J. Laaksonen, V. Vuori, E. Oja and J. Kangas, “Adaptation of Prototype Sets In On-line Recognition Of Isolated Handwritten Latin Characters”). The system also makes decisions driven by inter-class confusion rather than specific class-based allograph identification. The approach is also based on a binary decision, with the confusing prototypes “removed from the set of active prototypes”.
L. Schomaker, H. Teulings, E. Helsper, and G. Abbink, “Adaptive Recognition Of Online, Cursive Handwriting”, Proceedings of the Sixth International Conference on Handwriting and Drawing. Paris, Jul. 4-7, 1993: Telecom, (pp. 19-21) and L. Schomaker, H. Teulings, G. Abbink, and E. Helsper “Adaptive Recognition of On-line Connected-cursive Script for use in Pen-based Notebook Computers.” Handout, distributed with demonstrations presented at the IWFHR III, CEDAR, SUNY Buffalo, USA, May 25-27, 1993 describe a cursive script recognition system based on prototypical strokes clustered using a Kohonen Self Organising Map (SOM). The processes described use a writer-independent stroke-transition network that is used to recognise the written input.
If any of the top twenty possible words output by the recogniser are found to be a valid word as defined by a dictionary, the values in the stroke-transition network are “incremented in small steps until either the target word is at the top of the output list of words, or until a maximum number of iterations is reached.” The adapted system contains user-specific probabilities for individual stroke interpretations in the stroke-transition network.
L. Heutte, T. Paquet, A. Nosary and C. Hernoux, “Defining Writer's Invariants To Adapt the Recognition Task”, Proceedings of the Fifth International Conference on Document Analysis and Recognition, IEEE Computer Society, 1998 uses morphological writer-specific invariants to improve the recognition of an Offline Character Recognition (OCR) system. The technique requires the detection and clustering of writer-specific invariants, which are used in combination with contextual knowledge, to disambiguate the recognition process.
A number of approaches have been proposed that use adaptive techniques for the training of classifiers such as S. Connell and A. Jain, “Leaming Prototypes For On-Line Handwritten Digits”, Proceedings of the 14th International Conference on Pattern Recognition, Brisbane, Australia, pp. 182-184, August 1998, and G. Hinton, C. Williams, and M. Revow, “Adaptive elastic models for character recognition”, Advances in Neural Information Processing Systems 4, Morgan Kaufmann, San Mateo, Calif.
Similarly S. Connell and A. K. Jain, “Writer Adaptation of Online Handwritten Models,” Proc. 5th International Conference on Document Analysis and Recognition, Bangalore, India, pp. 434-437, September 1999 describes a system of writer adaptation that attempts to construct a user-specific writing style based on the identification of lexemes within a writer-independent model, and then uses this writer-dependent model to retrain their classifier. Since the approach is based around the training of a Hidden Markov Model (HMM) classifier, the adaptation technique is presumably performed once only for each writer (due to the large overhead required to retrain an HMM system). None of these techniques attempt to perform continuous user-specific adaptation based on user input.
U.S. Pat. No. 6,038,343, describes an approach to adaptation that uses the generation of writer-specific feature vectors in combination with a user-independent “universal recognition dictionary” to improve recognition results. They employ statistical regression to “predict a writer-specific feature vector for each of multiple alternative categories from the feature vector of an input character”. This method “corrects” the user-independent feature vectors by combining them with the writer-specific feature vectors, which are then used generatively to create character predictions for future user input vectors.
U.S. Pat. No. 5,917,924 uses adaptive weights that modify the likelihood of prototypes being matched as candidates in the recognition process and “only varies the weighting values in editing mode”. That is, the method only makes changes to the prototype weights based on results determined from user interaction.
U.S. Pat. No. 5,754,686 describes an approach to using a user-specific dictionary to store writer-dependent models. “If recognition reliability is low, a warning is issued. In response to a warning, the user or operator can decide whether the character pattern should be registered in the user dictionary.” The pattern is automatically assessed for suitability for inclusion in the user dictionary (presumably using a metric of ambiguity with existing prototypes) but no claim is made as to how the user-specific dictionary prototypes are combined with the existing writer-independent models during recognition.
U.S. Pat. No. 6,256,410 describes a standard scheme for training a writer-dependent HMM classifier, whereby user-specific training data is segmented using writer-independent models, and a set of character-based models are iteratively trained using the training data.
U.S. Pat. No. 5,319,721 describes a method for evolving a set of user-independent prototypes into a writer-dependent set. If the input stroke data matches a prototype to within a certain threshold, the user data is merged with the existing prototype to produce a writer-dependent prototype and “one or more starter prototype symbols of the same label as the input symbol” are deleted. If the stroke data does not match the existing prototypes, a new prototype is created, and again, one or more starter prototypes of the same label are deleted.
In U.S. Pat. No. 5,544,260 it is described that using “information provided during error correction for modifying character prototypes,” that is, using correction strokes made by the user to update mis-recognised prototypes. Similarly, U.S. Pat. No. 5,550,930 describes a method of storing recognition results, and, when requested by a user, displaying the results and allowing the user to select the input and corresponding symbols for classifier training.
U.S. Pat. No. 5,285,505 describes a method for “creating character prototypes for improving recognition accuracy of similarly shaped characters” by emphasising the sections of the character that are critical for discrimination and de-emphasising sections that are similar between characters. This approach is targeted specifically in resolving two-class ambiguities, such as ‘g’/‘y’ and ‘A’/‘H’ confusion.